{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# OpenAI APIs - Completions\n",
    "\n",
    "RTP-LLM provides OpenAI-compatible APIs to enable a smooth transition from OpenAI services to self-hosted local models.\n",
    "This tutorial covers the following popular APIs:\n",
    "\n",
    "- `v1/chat/completions`\n",
    "\n",
    "Check out other tutorials to learn about [vision APIs](./openai_api_vision.ipynb) for vision-language models and [embedding APIs](./openai_api_embeddings.ipynb) for embedding models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Launch A Server\n",
    "\n",
    "Launch the server in your terminal and wait for it to initialize.\n",
    "\n",
    "export CUDA_VISIBLE_DEVICES=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import subprocess\n",
    "from rtp_llm.utils.util import wait_sever_done, stop_server\n",
    "port=8090\n",
    "server_process = subprocess.Popen(\n",
    "        [\"/opt/conda310/bin/python\", \"-m\", \"rtp_llm.start_server\",\n",
    "         \"--checkpoint_path=/mnt/nas1/hf/models--Qwen--Qwen1.5-0.5B-Chat/snapshots/6114e9c18dac0042fa90925f03b046734369472f/\",\n",
    "         \"--model_type=qwen_2\",\n",
    "         f\"--start_port={port}\"\n",
    "         ]\n",
    "    )\n",
    "wait_sever_done(server_process, port)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chat Completions\n",
    "\n",
    "### Usage\n",
    "\n",
    "The server fully implements the OpenAI API.\n",
    "It will automatically apply the chat template specified in the Hugging Face tokenizer, if one is available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "\n",
    "port=8090\n",
    "client = openai.Client(base_url=f\"http://127.0.0.1:{port}/v1/chat/completions\", api_key=\"None\")\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "    model=\"qwen/qwen2.5-0.5b-instruct\",\n",
    "    messages=[\n",
    "        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"},\n",
    "    ],\n",
    "    temperature=0,\n",
    "    max_tokens=64,\n",
    ")\n",
    "\n",
    "print(f\"Response: {response}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parameters\n",
    "\n",
    "The chat completions API accepts OpenAI Chat Completions API's parameters. Refer to [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create) for more details.\n",
    "\n",
    "RTP-LLM extends the standard API with the `extra_configs` parameter, allowing for additional customization. One key option within `extra_configs` is `chat_template_kwargs`, which can be used to pass arguments to the chat template processor.\n",
    "\n",
    "#### Enabling Model Thinking/Reasoning\n",
    "\n",
    "```python\n",
    "# Ensure the server is launched with a compatible reasoning parser\n",
    "\n",
    "from openai import OpenAI\n",
    "\n",
    "port = 8090\n",
    "client = openai.Client(base_url=f\"http://127.0.0.1:{port}/v1\", api_key=\"None\")\n",
    "\n",
    "model = \"QwQ/Qwen3-32B-250415\" # Use the model loaded by the server\n",
    "messages = [{\"role\": \"user\", \"content\": \"9.11 and 9.8, which is greater?\"}]\n",
    "\n",
    "response = client.chat.completions.create(\n",
    "    model=model,\n",
    "    messages=messages,\n",
    "    extra_body={\n",
    "        \"extra_configs\": {\n",
    "            \"chat_template_kwargs\": {\n",
    "                \"enable_thinking\": True\n",
    "            }\n",
    "        },\n",
    "    }\n",
    ")\n",
    "\n",
    "print(\"response.choices[0].message.reasoning_content: \\n\", response.choices[0].message.reasoning_content)\n",
    "print(\"response.choices[0].message.content: \\n\", response.choices[0].message.content)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.chat.completions.create(\n",
    "    model=\"qwen/qwen2.5-0.5b-instruct\",\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"system\",\n",
    "            \"content\": \"You are a knowledgeable historian who provides concise responses.\",\n",
    "        },\n",
    "        {\"role\": \"user\", \"content\": \"Tell me about ancient Rome\"},\n",
    "        {\n",
    "            \"role\": \"assistant\",\n",
    "            \"content\": \"Ancient Rome was a civilization centered in Italy.\",\n",
    "        },\n",
    "        {\"role\": \"user\", \"content\": \"What were their major achievements?\"},\n",
    "    ],\n",
    "    temperature=0.3,  # Lower temperature for more focused responses\n",
    "    max_tokens=128,  # Reasonable length for a concise response\n",
    "    top_p=0.95,  # Slightly higher for better fluency\n",
    ")\n",
    "\n",
    "print(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Completions\n",
    "\n",
    "### Usage\n",
    "Completions API is similar to Chat Completions API, but without the `messages` parameter or chat templates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.completions.create(\n",
    "    model=\"qwen/qwen2.5-0.5b-instruct\",\n",
    "    prompt=\"List 3 countries and their capitals.\",\n",
    "    temperature=0,\n",
    "    max_tokens=64,\n",
    "    n=1,\n",
    "    stop=None,\n",
    ")\n",
    "\n",
    "print(f\"Response: {response}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parameters\n",
    "\n",
    "The completions API accepts OpenAI Completions API's parameters.  Refer to [OpenAI Completions API](https://platform.openai.com/docs/api-reference/completions/create) for more details.\n",
    "\n",
    "Here is an example of a detailed completions request:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.completions.create(\n",
    "    model=\"qwen/qwen2.5-0.5b-instruct\",\n",
    "    prompt=\"Write a short story about a space explorer.\",\n",
    "    temperature=0.7,  # Moderate temperature for creative writing\n",
    "    max_tokens=150,  # Longer response for a story\n",
    "    top_p=0.9,  # Balanced diversity in word choice\n",
    ")\n",
    "\n",
    "print(f\"Response: {response}\")"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
